Skip to content

fix(ai): derive safe model output token limits - #1632

Merged
wsp1911 merged 1 commit into
GCWing:mainfrom
wsp1911:dev
Jul 19, 2026
Merged

fix(ai): derive safe model output token limits#1632
wsp1911 merged 1 commit into
GCWing:mainfrom
wsp1911:dev

Conversation

@wsp1911

@wsp1911 wsp1911 commented Jul 19, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Hide maximum output Tokens from the model configuration UI while preserving existing explicit max_tokens overrides.
  • Derive automatic output limits from context-window tiers: 8K, 16K, 24K, 32K, and 64K.
  • Reject context windows below 32K and fall back to the automatic limit with a warning when an explicit output limit exceeds 40% of the context window.
  • Use the resolved output limit consistently for provider requests and context-compression budgeting.

Fixes #1558

Type and Areas

Type:

Bug fix, UI/UX

Areas:

Rust core, web UI, AI model configuration

Motivation / Impact

Users could set maximum output Tokens equal to the context window without understanding that BitFun reserves output capacity before triggering context compression. This left almost no input budget and caused repeated compaction.

The form now only asks for the context window. Existing manual max_tokens values remain supported when within 40% of the configured context window; invalid values safely fall back at runtime without rewriting user configuration.

Verification

  • Passed: pnpm run type-check:web
  • Passed: pnpm run i18n:audit
  • Passed: pnpm run fmt:rs
  • Passed: cargo test -p bitfun-core --lib derives_the_largest_output_tier_within_one_quarter_of_context -- --nocapture
  • Passed: cargo test -p bitfun-core --lib falls_back_to_the_automatic_output_limit_when_configured_limit_is_too_large -- --nocapture
  • Passed: cargo test -p bitfun-core --lib preserves_a_configured_output_limit_within_forty_percent_of_context -- --nocapture
  • Passed: cargo test -p bitfun-core --lib rejects_a_context_window_smaller_than_the_supported_minimum -- --nocapture
  • Passed: cargo test -p bitfun-core --lib rejects_a_model_context_window_smaller_than_32k -- --nocapture
  • Passed: cargo test -p bitfun-core --lib compression_trigger_budget_uses_the_automatic_output_tier_when_max_tokens_is_unset -- --nocapture
  • Passed: git diff --cached --check

The focused tests were run before the final data-only addition of the 24K automatic tier; no dedicated test was added for that tier by design.

Reviewer Notes

  • Runtime resolution produces one effective output limit shared by request construction and compression budgeting.
  • The UI preserves existing per-model overrides but does not copy hidden overrides to newly added models.
  • Invalid manual output limits log an English warning and fall back in memory; persisted configuration is not rewritten.
  • The serial Rust test runs passed; an earlier parallel build attempt hit an LLVM out-of-memory failure and was not treated as test success.

Checklist

  • This PR is focused and does not include secrets, temporary prompts, generated scratch files, or unrelated artifacts.
  • Relevant verification is recorded above, or skipped checks are explained.
  • User-facing strings, docs, and locales are updated where applicable.

Hide output-token configuration from the model form while preserving
existing explicit overrides. Derive a shared output limit for model
requests and context compression when the value is absent or invalid.

Reject context windows below 32K and fall back with a warning when an
explicit output limit exceeds 40% of the context window.

Refs GCWing#1558
@wsp1911
wsp1911 marked this pull request as ready for review July 19, 2026 10:44
@wsp1911
wsp1911 merged commit 4f55608 into GCWing:main Jul 19, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: 无限压缩

1 participant